Limited error raster compression

ABSTRACT

Embodiments of the invention relate to an image or raster compression method that includes receiving pixel data for a raster comprising a two dimensional (2D) array of pixels where each pixel is associated with a data value. The method further includes receiving a user defined parameter defining a maximum error allowable per pixel for a compression algorithm. The raster can be divided into a number of pixel blocks where each pixel can be quantized and bit stuffed based on a number of block statistics including the maximum error allowable. The method further includes executing the compression algorithm wherein for each pixel, where an error caused by the compression algorithm is equal to or less than the maximum error allowable, and encoding the pixel data based on the compression algorithm. In certain embodiments, the compression algorithm is a non-transform compression algorithm.

CROSS-REFERENCES TO RELATED APPLICATIONS

NOT APPLICABLE

BACKGROUND OF THE INVENTION

Conventional topographical surface mapping technologies that have beenused to create elevation rasters, maps, and images, tend to approximatethe surface of the earth relatively coarsely. For example, DTED-2(Digital Terrain Elevation Data) format can use spatial resolutions(i.e., different cell sizes) of 30×30 meters with a vertical resolutionof 1 meter. The data is typically stored as a 16 bit integer raster (−32km-+32 km), which provides enough range to represent any point on earth,but not a high enough resolution to map many above ground features(i.e., trees, buildings, etc.) due to the large cell sizes.

Contemporary surface mapping systems can provide highly detailed imageswith cell sizes smaller than one square meter and vertical resolutionsbelow 1 cm. These systems can create images rich in detail and contentincluding buildings, cars, brush, trees, or other surface features giventhe small cell size and shallow vertical resolution. As a result, theseimages can result in very large data files that render conventionalcompression technologies (e.g., LZW) ineffective. In many modernsystems, users have the choice between lossless compression (e.g., LZW)on the one hand, which can be ineffective on high detail, highly noisydata, and lossy compression methods (e.g., jpg) on the other hand, whichcan produce arbitrarily large compression errors for a single pixel.Better compression methods are needed that can be lossy, and provideadequate compression ratios for contemporary high resolution rasterfiles.

SUMMARY OF THE INVENTION

Embodiments of the invention related to a compression method includingreceiving pixel data representing a two dimensional array of pixels,where each pixel is associated with a data value and receiving a firstuser defined parameter defining a maximum error allowable per pixel fora compression processing routine. The method can further includeexecuting the compression processing routine, where the data value foreach pixel is compressed using the compression processing routine, andwhere an error caused by the compression processing routine is equal toor less than the maximum error allowable, and encoding the pixel databased on the compression processing routine. In some cases, the pixeldata can be larger than 8 bit, and can be at least one of scientificdata, elevation data, or satellite imagery, etc.

In some embodiments, the compression processing routine can be a notransform compression processing routine that includes quantizing thepixel data into an array of unsigned integer values, and bit stuffingthe array of unsigned integers into a byte array. The method alsoincludes dividing the 2D array of pixels into pixel blocks, whereencoding the pixel data further includes encoding each of the pixelblocks. Dividing the 2D array of pixels into pixel blocks can furtherinclude determining a set of statistics from each block, where the setof statistics includes a maximum and minimum value in each block and anumber of valid pixels in each block, and computing the number of bytesrequired to maximally compress each of the blocks based on the set ofstatistics. In some cases, the encoding of each of the blocks occursonly once.

Certain embodiments of the invention are related to a non-transitorycomputer-readable storage medium including a plurality ofcomputer-readable instructions tangibly embodied on thecomputer-readable storage medium, which, when executed by a dataprocessor, provides a lossy compression method, the plurality ofinstructions including instructions that cause the data processor toreceive pixel data representing a two dimensional array of pixels,wherein each pixel is associated with a data value and receive a firstuser defined parameter defining a maximum error allowable per pixel fora compression processing routine. Further embodiments can includeinstructions that cause the data processor to execute the compressionprocessing routine (e.g., an algorithm), where the data value for eachpixel is compressed using the compression processing routine, and wherean error caused by the compression processing routine is equal to orless than the maximum error allowable. Yet further embodiments caninclude instructions that cause the data processor to encode the pixeldata based on the compression processing routine. In some cases, thepixel data can be at least 8 bit or larger, and can be at least one ofscientific data, elevation data, or satellite imagery, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A depicts an aircraft utilizing LiDAR technology to scan andsurvey the terrain below.

FIG. 1B depicts an aircraft utilizing LiDAR technology and receivingmultiple reflections from a single set of planar coordinates.

FIG. 2A illustrates a LiDAR-generated elevation raster depicting alandscape with a MaxZError threshold of 1 meter.

FIG. 2B illustrates a LiDAR-generated elevation raster depicting alandscape with a MaxZError threshold of 10 centimeters.

FIG. 3 depicts a simplified flow diagram illustrating aspects of amethod of compression, according to embodiments of the invention.

FIG. 4 depicts a simplified flow diagram illustrating aspects of amethod of compression, according to embodiments of the invention.

FIG. 5 depicts a simplified flow diagram illustrating aspects of amethod of encoding one block in a compression scheme, according tocertain embodiments of the invention.

FIG. 6 depicts a simplified flow diagram illustrating aspects of amethod of determining an optimal block size for a compression scheme,according to certain embodiments of the invention.

FIG. 7A depicts a tile including a plurality of uniformly sized pixelblocks, according to an embodiment of the invention.

FIG. 7B depicts a raster including a plurality of tiles, according to anembodiment of the invention.

FIG. 8A depicts a LERC file structure, according to an embodiment of theinvention.

FIG. 8B depicts a LERC file header, according to an embodiment of theinvention.

FIG. 8C depicts a LERC mask or pixel values header, according to anembodiment of the invention.

FIG. 8D depicts a LERC block header, according to an embodiment of theinvention.

FIG. 9A depicts sample block data for a 4×4 block of pixels, accordingto an embodiment of the invention.

FIG. 9B depicts quantized sample block data for a 4×4 block of pixels,according to an embodiment of the invention.

FIG. 9C depicts a sample block header, according to an embodiment of theinvention.

FIG. 9D depicts quantized sample block data for a 4×4 block of pixels,according to an embodiment of the invention.

FIG. 9E depicts a sample block header, according to an embodiment of theinvention.

FIG. 10 illustrates a computer system, according to an embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention can include a Limited Error RasterCompression (LERC) method that includes receiving pixel data for araster comprising an array of three-dimensional point data where eachpixel is associated with a data value. The method further includesreceiving a user defined parameter defining a maximum error allowablefor a compression algorithm. The raster can be divided into a number ofpixel blocks where each pixel can be quantized and bit stuffed based ona number of block statistics including the maximum error allowable. Themethod further includes executing the compression algorithm wherein foreach pixel, where an error caused by the compression algorithm is equalto or less than the maximum error allowable, and encoding the pixel databased on the compression algorithm.

In some instances, raster data with very large pixel depth (e.g., morethan 8 bit per pixel) can comprise very large data files, which can bedifficult to transfer or process in real-time. For example, 8-bit datacan include numbers ranging from 0-255, 16-bit data can include numbersranging from 0-65535, and so on, for each individual pixel of a givenraster. Typically, lossless compression methods try to assign smallercodes to symbols with higher occurrences, and larger codes to symbolswith fewer occurrences. This can be problematic when there are largenumbers of different data values which may result in overall losslesscompression factors at 2× or less.

Raster data can include scientific raster data (e.g., temperature,biomass production, evaporation, transpiration, air pressure, etc.),multi-spectral imagery, or elevation raster data (e.g., 2D raster withelevation data in each cell), or the like, which can be derived from 3Dpoint cloud data. One example of three-dimensional point cloud dataincludes Light Detection and Ranging (LiDAR) data. LiDAR is an opticalremote sensing technology that can measure the distance to a target byilluminating the target with light (e.g., pulses from a laser), whichcan be used to map physical features (e.g., geographic, microscopic,etc.) with very high resolution. LiDAR technology can be fitted toaircraft or satellites for surveying and mapping geographic areas. FIG.1A depicts an aircraft 100 utilizing LiDAR technology to scan and surveythe terrain below. The data is collected by a laser unit 110 attached tothe aircraft that emits pulses toward the ground in a sweeping pattern120. The laser unit 110 receives the reflections and generates a veryhighly detailed 3D point cloud raster (i.e., map) of the terrain below,with image resolutions that can far surpass the capabilities of earliertechnologies. The LiDAR data files are typically very large, makingreal-time streaming high resolution data problematic over bandwidthlimited mediums.

The raw output from a LiDAR system can be a 3D point cloud, which can bea cluster of points in 3D space with coordinates [x, y, z]. Those pointscan be stored in LAS files, which are a public format allowing theexchange of LiDAR data between the organizations and companies whoproduce this data and those who consume it. Note that for a point cloud,there can be multiple points on top of each other, having the same [x,y] coordinates but different [z] coordinates. To illustrate, FIG. 1Bdepicts an aircraft 100 utilizing LiDAR technology 110 and receivingmultiple reflections (122, 124, 126, 128) from a single set of planarcoordinates. A single laser pulse can result in multiple returns, whichmay return at different intervals. For example, the first return 122 canbe from a leaf at the top of a tree, while the last return 126 can befrom the reflection of the laser pulse off of the ground. From thetravel time of the pulse and the 3D position of the LiDAR unit on theplane (GPS), the laser pulse reflection points can be measured with anaccuracy on the order of centimeters, depending on multiple factors suchas the height of the plane and the accuracy of the LiDAR measurements.Additionally, each point (i.e., coordinate) can have a flag attached(e.g., 1st, 2nd, . . . , last return, with classification flags (ascomputed by classification algorithms) such as ground, water, building,etc.). From the flagged data points, different elevation surfaces can beindividually reconstructed. For instance, bare earth or ground surface(buildings and trees are removed) can be reconstructed by using 3Dpoints that have the “ground” flag set. In some cases, this surface willinitially have holes or void areas where other features (e.g., trees,buildings, etc.) were. Similarly, a tree canopy surface can bereconstructed by using 3D points that have a “first return” flag set. Insome embodiments, these surface rasters can be computed using eitherbinning (e.g., aggregation of all points in each raster cell based ontheir average or maximum value per cell), triangulation (i.e.,interpolation between measured points), or other method known by thoseskilled in the art.

There are a variety of ways to derive a 3D map or elevation raster froma 3D point cloud. In one non-limiting example, the 3D point cloud isoverlaid with a regular [xy] grid and all of the points in each gridcell are combined into one value, as described above. This can be doneby computing the average [z] or height of all points in each cell.Alternatively, the minimum or maximum [z] can be used. Furthermore, allpoints can be connected to create a mesh using triangulation, where anintersection is made at the cell centers resulting is one [z] value pergrid cell. Further still, one can define which points of the entire 3Dpoint cloud should be included in this process. For example, this mightbe limited to points labeled as “ground,” “vegetation,” “buildings,”etc., as described above. It should be noted that in certain embodimentsof the compression method described herein, it is assumed that the 3DLiDAR point cloud has already been converted into one elevation rasterusing any of the previously mentioned methods or other methods known bythose of ordinary skill in the art. In short, the compression methoddescribed herein starts with a raster or an image with N rows, Mcolumns, and one z or elevation value for each grid cell.

FIG. 3 depicts a simplified flow diagram illustrating aspects of amethod 300 of compression, according to embodiments of the invention.The LERC method 300 includes receiving pixel data representing atwo-dimensional array of pixels, where each pixel is associated with adata value (310). In some embodiments, the raster data can includescientific raster data (e.g., temperature), multi-spectral imagery,elevation raster data (i.e., 2D raster with elevation data in eachpixel/cell), and the like. As described above, raster data can have alarge pixel depth (e.g., more than 8 bit per pixel), which can bedifficult to compress using conventional lossless methods.

At 320, the method includes receiving a user defined maximum error(i.e., “MaxZError”) allowable per pixel. The MaxZError value determinesthe amount of error introduced when quantizing pixel data values duringcompression, as further discussed below with respect to FIG. 4. The usercan select a maximum allowable error based on a desired data precision.Allowing greater error will typically yield greater compression ratiosand reduced file sizes. For example, a maximum error of 0.1 cm in thez-direction can have significantly less error (and a greater file size)than a maximum allowable error of 1 meter. Lower MaxZError values alsoproduces, for example, elevation rasters with higher precision thanhigher MaxZError values. For example, FIG. 2A illustrates aLiDAR-generated elevation raster depicting a landscape with a MaxZErrorvalue (i.e., threshold) of 1 meter. FIG. 2B illustrates the samelandscape with a MaxZError of 10 cm. Where FIG. 2A shows “blocky” orstep-like gradients in mountainous regions, FIG. 2B shows those samefeatures with a much smoother gradient because of the smaller errorintroduced during compression. In these examples, the compression ratiosachieved are 24.2 for FIGS. 2 a and 8.5 for FIG. 2 b, in contrast tocompression ratios as low as 1.7 when using standard lossless LZWcompression. In addition, LERC encoding speed can be approximately 10×faster than LZW or more. It should be noted that although many examplesherein refer to the pixel data value as a vertical displacement(z-axis), the pixel data value can be any type of data includingtemperature, force, mass, luminescence, or any other desired parameterthat would apply to any raster, data, and the like.

At 330, the method includes executing the compression algorithm tocompress the data value for each pixel within the user defined maximumerror. In other words, the data value for each pixel can be maximallycompressed while maintaining a quantization error below the user definedmaximum (i.e., MaxZError). The method further includes encoding thepixel data based on the output of the compression algorithm (340). Thecompression algorithm (i.e., processing routine) can be a no-transformcompression processing algorithm. It should be noted that thecompression algorithms described herein do not include discrete cosinetransforms (DCT) or wavelet transforms. The individual steps thatcomprise the compression algorithm are further described below withrespect to FIG. 4.

FIG. 4 depicts a simplified flow diagram illustrating aspects of amethod 400 of compression, according to embodiments of the invention.The LERC method 400 includes receiving pixel data representing atwo-dimensional array of pixels, where each pixel is associated with adata value (410). In some embodiments, the raster data can includescientific raster data (e.g., temperature), multi-spectral imagery,elevation data, etc. It should further be noted that LERC can beextended to three-dimensional voxel grids (e.g., MRI, CT, etc.).

At 420, the LERC method 400 includes receiving a user defined maximumerror (i.e., “MaxZError”) allowable per pixel. The MaxZError valuedetermines the amount of error introduced when quantizing pixel datavalues during compression. The user can select a maximum allowable errorbased on a desired data precision. Allowing greater error will typicallyyield greater compression ratios and reduced file sizes, and vice versa.

At 430, the method 400 continues with searching for and determining theoptimal block size for subdividing the image into pixel blocks, asfurther described below with respect to FIG. 6. At 440, for the optimalblock size, dividing the image into pixel blocks and encoding each pixelblock, as further described below with respect to FIG. 5.

To further illustrate the LERC compression method by example, thecompression algorithm includes dividing the raster into pixel blocks(e.g., 8×8, 11×11, or larger blocks). The smaller the block, the smallerthe value range of the pixel values, resulting in less bits needed perpixel. Each block stores its own data (e.g., min value for that block,the number of bits needed per pixel, etc.). The optimal block sizedepends on that data and is determined by the compression algorithm. Thequantization component of the compression algorithm takes a block ofpixels (e.g., 8×8 pixels=64 pixels) and determines the minimum andmaximum pixel value in that particular block. Keeping the minimum valueas an offset for this block, the 64 pixel values can be quantized basedon the specified maximum error allowed (MaxZError) per pixel. As aresult, each value can be represented needing only, for instance, 10bits instead of 32 bits (e.g., as required for floating point numbers).These 64 by 10 bits can be “bit stuffed” into an array of the sizeneeded (e.g., 640 bits or 80 bytes). Some non-limiting, exemplaryembodiments of the LERC method are presented below in FIGS. 9A-9E. Incertain embodiments, the compression algorithm (i.e., compressionprocessing routine) is not a discrete cosine transform (DCT) or wavelettransform.

FIG. 5 depicts a simplified flow diagram illustrating aspects of amethod 500 of encoding one block in a compression scheme, according tocertain embodiments of the invention. The compression method 500includes computing a minimum and maximum data value in a pixel block andthe number of valid pixels (510). At 520, if all of the pixel block datais either invalid, or all 0's, 1's, or −1's, the block is encoded as asingle byte (525) and the method ends. At 530, if the pixel blockincludes a constant block (i.e., all uniform pixel values) that are notall invalid or all 0's, 1's, or −1's, the pixel block is encoded with aheader with no data (535) and the method ends. File structures, fileheaders, block headers, and pixel value headers can be arranged andconfigured in any suitable format as would be known and appreciated byone of ordinary skill in the art. Some exemplary embodiments of filestructures are shown in FIGS. 8A-D.

At 540, the method includes determining whether the pixel block shouldbe encoded uncompressed. This may occur if the MaxZError=0 or verysmall, as can be determined by equation (1).

((Max−Min)/(2*MaxZError)>2²⁸)  (1)

According to equation (1), if more than 28 bits per pixel are requiredto encode the quantized integers, then the block is not compressed andremains as is in float. It should be noted that equation (1) is but oneexemplary embodiment and other formulas may be used to determine whenLERC compression provides an inadequate compression ratio for aparticular application. In such cases, the method further includesencoding the block lossless (545) and ending. Typically, most pixelblocks are suitable for compression. In these cases, the method furtherincludes determining the minimum pixel data value in the pixel block andstoring it as an offset in the block header (550). One example of ablock header is shown in FIG. 8D, although other header formats may beused. The method further includes quantizing pixel values according tothe quantization algorithm (560) shown in equation (2). It should benoted that the quantization process is the only lossy step in thecompression algorithm described herein.

n(i)=(unsignedint)((x(i)−min)/(2*maxZError)+0.5)  (2)

In this case, n(i) is a new integer value after quantization and x(i) isthe value before quantization. The x(i) value can be float or integer atpixel (i) and n(i) can be unsigned long (e.g., non-negative 32-bitinteger). At 570, the LERC method further includes computing the numberof bits needed to encode the full range of non-negative integerslossless and bit stuffing the integers into a byte array. In someembodiments, the block header is updated with the number of bits perpixel used and the number of valid pixels (580), as shown in FIG. 8D.

FIG. 6 depicts a simplified flow diagram illustrating aspects of amethod 600 of determining an optimal block size for a compressionscheme, according to certain embodiments of the invention. The LERCmethod 600 includes beginning a search loop over successive prospectivepixel block sizes (610). For example, the method 600 may begin withsuccessively looping over the entire raster image and subsets thereof.In one non-limiting example, the method 600 may start with the entireraster image and then continuing from the smallest pixel block (e.g.,8×8) and increasing therefrom with each iteration (e.g., pixel blocksizes of 8, 11, 15, 20, 32, etc.). In another embodiment, the method maybegin by iteratively looping through increasing pixel block sizesstarting with a small pixel block and progressively increasing. For eachprospective block size, the method further includes computing theminimum (Min) and maximum (Max) pixel data values and the number ofvalid pixels for that block size (620) for each block in the raster. Asdescribed above, the pixel data can include elevation data, temperaturedata, or any other data associated with one or more coordinates in araster image.

At 630, the method includes computing the number of bytes needed toencode the current pixel block using each of the Min, Max, the number ofvalid pixels, and the user defined MaxZError parameters. As shown inFIG. 6, process steps 620 and 630 are performed for each block of theraster for the current block size. At 640, the method continues withcomputing the number of bytes needed to encode the entire raster image(i.e., all blocks) for the computed block size. For example, a rasterimage may be large enough to accommodate 36 pixel blocks, each with an8×8 pixel array. Other cases may accommodate fewer or larger numbers ofpixel blocks depending on the size of the raster and/or pixel blocks. At650, if the block size is one of a number of valid block sizes (e.g.,11, 15, 20, 32, 64) and the number of bytes for that particular blocksize has increased compared to the previous computed block size, thesearch loop ends early (655), the block size with the smallest number ofbytes is selected (660), and the method ends. At 650, if the block sizeis one of the number of valid block sizes and the number of bytes forthat particular block size has not increased compared to the previouscomputed block size, the method continues by determining whether thecurrent block size is less than 64×64 pixels (660). If the block size isless than 64, the next higher block size is selected at the methodreturns to 620. If the current block size is not less than 64 (e.g., isequal to 64×64 pixels), the method continues with selecting the blocksize utilizing the smallest number of bytes (660).

In summary, the method 600 provides an efficient and systematic approachfor determining the optimal block size for a particular raster. In someembodiments, as long as the calculated byte size continues to improvewith each successive block size (i.e., require less bytes), the methodwill continue until the smallest number of bytes is determined. If thebyte size for a particular block increases as compared to the previousblock, then it may be assumed that the minimum byte size has beencrossed and the method can stop. This method may allow the minimum bytesize to be determined before actually cycling through every availableblock size, which may improve the overall efficiency of a system (e.g.,processor 1030 of FIG. 10) executing the method 600.

It should be appreciated that the specific steps illustrated in FIG. 6provides a particular method 600 of compression, according to anembodiment of the present invention. Other sequences of steps may alsobe performed according to alternative embodiments. In certainembodiments, the method 600 may perform the individual steps in adifferent order, at the same time, or any other sequence for aparticular application. Moreover, the individual steps illustrated inFIG. 6 may include multiple sub-steps that may be performed in varioussequences as appropriate to the individual step. Furthermore, additionalsteps may be added or removed depending on the particular applications.One of ordinary skill in the art would recognize and appreciate manyvariation, modification, and alternatives of the method.

FIG. 7A depicts a tile 700 including a plurality of uniformly sizedpixel blocks 710, according to an embodiment of the invention. Eachblock 710 is uniform and compressed to produce the smallest number ofbytes per block, as described above with respect to FIG. 6. FIG. 7Bdepicts a raster 750 including a plurality of tiles, according to anembodiment of the invention. Some raster images may be very large (e.g.,30K pixels or more) and can be stored as a sequence of tiles (e.g, .tiffile), where each tile is encoded separately using different blocksizes. Raster 750 includes tile 770, 775, 780, and 785. Each tile canhave a different uniform block size generated from a differentcompression scheme. For example, tile 770 has larger blocks 772 than theblocks 777 of tile 775. In some cases, dividing raster images intomultiple tiles can yield smaller overall raster data files. For example,if tile 775 was a LiDAR scan of mountainous terrain, each pixel blockwould be relatively small to accommodate a large variation in elevation(i.e., min and max pixel data values). Conversely, if tile 785 was aLiDAR scan of a lake, each pixel block would be larger given the littleto no variation in elevation, resulting in a small number of pixelblocks. Typically, tiles with large numbers of blocks tend to yieldrelatively larger data files than tiles with a smaller number of pixelblocks.

File Structures

FIG. 8A depicts a LERC file structure, according to an embodiment of theinvention. The LERC file structure can include a file header field, amask header field, mask data field, a pixel values header field, andpixel values data field as well as their associated formats and sizes.Other data fields can be included in the file structure that may differin data type or size as required. FIG. 8B depicts a LERC file header,according to an embodiment of the invention. The LERC file header caninclude a file identifier string, a file version, an image type, animage height in pixels, an image width in pixels, and the user selectedMaxZError, including their associated formats and sizes. FIG. 8C depictsa LERC mask or pixel values header, according to an embodiment of theinvention. The LERC mask or pixel values header includes the number ofblocks in the vertical direction, the number of blocks in the horizontaldirection, the number of bytes, and a maximum value in the image (e.g.,the or raster), as well as their associated formats and sizes. FIG. 8Ddepicts a LERC block header, according to an embodiment of theinvention. The LERC block header includes the encoding type, the offset,the number of bits per pixel, and the number of valid pixels, includingtheir associated formats and sizes.

Utilizing the File Structures in LERC

In certain embodiments of the invention, the LERC method includesquantizing an array of floating numbers (e.g., elevation data,temperature data, etc.) into an array of unsigned or positive integersusing a user-specified maximal quantization error (i.e., MaxZError) anddetermining the minimum and maximum data value of the float array. Themethod includes storing the minimum data value as an offset value. Insome cases, the offset value is stored as a float number in a blockheader (FIG. 8D). In some embodiments, the conversion equation isn(i)=(x(i)−min)/(2*MaxZError)+0.5. In cases where the MaxZError is toosmall such that the maximum float array element cannot be represented asa 32 bit unsigned integer (e.g., MaxError=0), the float array is notconverted to integer and is stored as a float array uncompressed. Itshould be noted that the quantization step is the only lossy componentin the compression algorithm.

According to some embodiments, the method continues with bit stuffingthe array of unsigned integer numbers into an array of unsigned char orbytes. In some aspects, if the maximum array element needs m bits, thenthe entire array of N elements can be compressed lossless into a bytearray of length (N*m+7)/8 using low level bit shifting operations. Themethod further includes dividing the elevation raster into blocks andencoding each block.

The compression algorithm can automatically determine the optimal blocksize. Encoding the input raster multiple times using a series ofdifferent block sizes and choosing the block size resulting in thesmallest file size can require additional file resources and be verytime consuming. The LERC method can determine the optimal block sizeprior to encoding the data by first computing the number of bytes neededto encode each block using various block statistics (e.g., max and minvalue, number of valid pixels) prior to encoding the block. As such, themethod includes computing a resulting file size for a number ofdifferent block sizes and selecting the optimal block size to encode theraster a single time.

Typically, the smaller the block size (e.g., 8×8), the more important itbecomes to minimize the overhead needed to encode each block. In someembodiments, the block header (FIG. 8D) is defined as {encoding type(byte), offset (variable type), number of bits per element (byte),number of valid pixel (variable type)}. With the variable type, themethod dynamically switches to the smallest type that allowsrepresenting the information without loss. For example, for the lastfield of the header, “number of valid pixels,” only one byte is neededfor block sizes up to 15×15 pixels (15×15=225 pixels and fits into onebyte). Larger blocks may require unsigned short (2 bytes). In caseswhere the entire image can be encoded as one block, an unsigned integermay be utilized. As block sizes increase, the block overhead becomesless significant in the overall compression scheme. The variable typeused can be encoded in the highest 2 bits of the byte just before thatvariable. Similarly, if the offset value is a real float value, it canbe encoded as a float (4 bytes). If the offset value is a 16-bit integer(e.g., Digital Terrain Elevation Data (DTED)), it can be encoded as ashort (2 bytes). Alternatively, the offset value can be written andencoded as a char (1 byte). Using the two bytes located before theoffset and the number of valid pixels, the highest 2 bits can be used toencode the variable type of the following variable, and the lower 6 bitscan be used by these two bytes themselves. The number of bits perelement can be used to encode the bit depth used for bit stuffing thearray of unsigned integers, which is typically 32 bits or less. Thefirst byte of the header (i.e., the encoding type) is encoded toidentify whether the block is stored as lossless float, or bit stuffed,and in addition encodes common situations as just one byte. For example,a block of a void (no data) mask being constant (i.e., all valid or allinvalid), or an elevation block constant at 0, can be represented by theone byte, allowing frequently occurring blocks to be encoded as a singlebyte.

In some embodiments, a void or no data mask is encoded first and theelevation values thereafter. The void mask can be represented as binary(e.g., for valid, invalid data), integer (e.g., encode like pixel valueswith MaxZError=0.5 for lossless encoding), a float raster (e.g., withmask used as a weight kernel), or any desired data structure. In somecases, the void pixel can be set to avoid interpolations at −1 or may beinterpolated at 0 with valid pixels set to 1. Alternatively, the datamask can be used as a count (e.g., the number of LiDAR pulses hitting aparticular cell), a weight (e.g., float number between 0 and 1), or anyother desired data representation. To maintain flexibility, the voidmask and elevation raster can be stored as a float raster and can bothbe compressed using the methods described herein. In each case, the maskis typically encoded lossless. In certain embodiments, if the mask isbinary (valid/invalid), which can be the most common case, it can beencoded as follows: Put all bits into a byte array of length ((number ofpixels+7)/8) bytes, and run-length encode this array. The array willtypically become very small, particularly with large areas of all validor all invalid pixels. Even without run-length encoding, the mask takesonly 1 bit per pixel. In such cases, the binary mask contributes littleto the overall data volume and does not typically justify using moreexpensive or time consuming compression methods.

Before encoding a float raster, the method includes running throughdifferent block size configurations and computing both statistics andthe number of bytes needed to encode the raster using that particularblock size. Some typical block sizes may include {entire raster, 8, 11,15, 20, 32, 64}. In some embodiments, block sizes greater than 64×64 arenot used because at this block size the block overhead becomesnegligible and larger blocks typically do not yield better compressionoutput. In some cases it may be advantageous to start with the entireraster to allow for early stopping in special cases (e.g., entire rasteris constant, all valid, or all invalid).

As described above, once the optimal block size has been determined, theraster is encoded. In certain embodiments, the method includes encodingsingle blocks containing outliers as float numbers uncompressed and bitstuffing regular blocks. The variable types can allow for automaticswitching from float to integer wherever possible. Typically, the maskcontaining information identifying valid and invalid pixels is encodedfirst and only elevation values (or other ‘z’ data) for valid pixels areencoded. During the decoding process, the mask is typically decodedfirst in order to determine which pixels have to be filled with theelevation values being decoded.

In certain embodiments, alternatives to bit stuffing may be used tocompress the pixel raster. For example, instead of bit stuffing thenon-negative integers after quantization, all of the non-negativeintegers can be collected unchanged in one large array of non-negativeintegers for all blocks. This will provide the LERC header structurewith all block headers for all blocks, but no data therein, as well asone large array with all the non-negative integers from the differentquantizations of the entire image. Conventional lossless encoding can beapplied to the array of integers (e.g., symbol based encoding schemes(LZW), etc.). It should be noted that this scheme may produce some floatnumber from blocks that were not quantized but passed alonguncompressed. While this method can produce significant compressiongain, it can involve increased complexity and computation time for bothencoding and decoding processes.

Example of LERC Encoding for One Block Using MaxZError=0.01 m

In some embodiments, LERC encoding can be performed using foursimplified basic steps including, but not limited to, (1) calculatingthe basic statistics for a data block; (2) determining how to encode theblock using a user defined MaxZError value, (3) determining the numberof bits needed and bit stuffing the non-negative integers; and (4)writing the block header. FIG. 9A depicts sample block data 910 for a4×4 block of pixels, according to an embodiment of the invention. Itshould be noted that blocks are typically larger than 8×8 pixels, butare shown here as 4×4 pixels for simplicity of description. At step 1,the LERC method includes determining the basic statistics for the datablock of FIG. 9A. The minimum value is 1222.2943 and the maximum valueis 1280.8725. The number of valid pixels (i.e., non-void pixels) is 12of 16. The number of invalid pixels (i.e., void pixels) is 4 of 16.

At step 2, the LERC method includes determining whether the pixel blockshould be encoded uncompressed with a user-input MaxZError of 0.01 m. Byusing equation (1) above:(Max−Min)/(2*MaxZError)=>(1280.8725−1222.2943)/(2*0.01)=2,928.91. Sincethis value is less than 2²⁸, we can quantize the pixel values in datablock 910 and expect an acceptable compression ratio. The block isquantized using equation (2) above, where each new pixel value iscalculated by: n(i)=(unsigned int)((x(i)−Min)/(2*MaxZError)+0.5),resulting in the new pixel block 920 shown in FIG. 9B. At step (3), themethod further includes determining the number of bits needed and bitstuffing these non-negative integers. The number of bits needed can bedetermine by equation (3):

NumBits=ceil(log₂(2929))=12  (3)

To represent the number of bits needed another way, 2¹¹<2929<2¹². Inthis case, 12 bits per number are need to encode all numbers of block910 lossless. There are 12 valid numbers, resulting in 12×12=144 bitstotal. As 144/8=18, 18 bytes are needed to encode the entire data block.At step (4), the method includes writing the block header as shown inFIG. 9C. It follows that 7 bytes are needed for the block header. Thetotal number of bytes needed for block 920 can be calculated as 18+7=25bytes. In this particular case, the header takes too much space withrespect to the raw pixel data, which exemplifies the need to work withblock sizes of 8×8 pixels or larger.

Example of LERC Encoding for One Block Using MaxZError=1.0 m

Using the same 4×4 pixel block shown in FIG. 9A, the LERC method isperformed again using a larger MaxZError (i.e., error threshold) of 1.0m. Beginning with step (1), the minimum value is 1222.2943 and themaximum value is 1280.8725. The number of valid pixels (i.e., non-voidpixels) is 12 of 16. The number of invalid pixels (i.e., void pixels) is4 of 16. In step (2), the LERC method proceeds with determining whetherthe pixel block should be encoded uncompressed with a user-inputMaxZError of 1.0 m. As By using equation (1) above:(Max−Min)/(2*MaxZError)=>(1280.8725-1222.2943)/(2*1.0)=29.2891. Sincethis value is less than 2²⁸, we can quantize the pixel values in datablock 910 and expect an acceptable compression ratio. The block isquantized using equation (2) above, where each new pixel value iscalculated by: n(i)=(unsigned int)((x(i)−Min)/(2*MaxZError)+0.5),resulting in the new pixel block 940 shown in FIG. 9D. At step (3), themethod further includes determining the number of bits needed and bitstuffing these non-negative integers. The number of bits needed can bedetermine by equation (4):

NumBits=ceil(log₂(29))=5  (4)

To represent the number of bits needed another way, 2⁴<29<2⁵. In thiscase, 5 bits per number are need to encode all numbers of block 940lossless. There are 12 valid numbers, resulting in 5×12=60 bits total.As 60/8=7.5, 8 bytes are needed to encode the entire data block. At step(4), the method includes writing the block header as shown in FIG. 9E.It follows that 7 bytes are needed for the block header. The totalnumber of bytes needed for block 920 can be calculated as 8+7=15 bytes.

FIG. 10 illustrates a computer system 1000 according to an embodiment ofthe present invention. The compression methods described herein (e.g.,FIGS. 3-6) can be implemented within a computer system such as computersystem 1000 shown here. Computer system 1000 can be implemented as anyof various computing devices, including, e.g., a desktop or laptopcomputer, tablet computer, smart phone, personal data assistant (PDA),or any other type of computing device, not limited to any particularform factor. Computer system 1000 can include processing unit(s) 1030,storage subsystem 1010, input devices 1050 (e.g., keyboards, mice,touchscreens, etc.), output devices 1060 (e.g., displays, speakers,tactile output devices, etc.), network interface 1070 (e.g., RF, 4G,EDGE, WiFi, GPS, Ethernet, etc.), and bus 1005 to communicatively couplethe various elements of system 1000 to one another.

Processing unit(s) 1030 can include a single processor, multi-coreprocessor, or multiple processors and may execute instructions inhardware, firmware, or software, such as instructions stored in storagesubsystem 1010. The storage subsystem 1010 can include various memoryunits such as a system memory, a read-only memory (ROM), and permanentstorage device(s) (e.g., magnetic, solid state, or optical media, flashmemory, etc.). The ROM can store static data and instructions requiredby processing unit(s) 1030 and other modules of the system 1000. Thesystem memory can store some or all of the instructions and data thatthe processor needs at runtime.

In some embodiments, storage subsystem 1010 can store one or moresoftware programs to be executed by processing unit(s) 1030, such as thecompression algorithm 1012, the quantization algorithm 1014, or theblock optimization algorithm 1016, as further described above withrespect to FIGS. 3-6 and 9A-9E. As mentioned, “software” can refer tosequences of instructions that, when executed by processing unit(s)1030, cause computer system 1000 to perform certain operations of thesoftware programs. The instructions can be stored as firmware residingin read-only memory and/or applications stored in media storage that canbe read into memory for processing by processing unit(s) 1030. Softwarecan be implemented as a single program or a collection of separateprograms and can be stored in non-volatile storage and copied in wholeor in part to volatile working memory during program execution. Fromstorage subsystem 1010, processing unit(s) 1030 can retrieve programinstructions to execute in order to execute various operations (e.g.,compression algorithms) described herein.

It will be appreciated that computer system 1000 is illustrative andthat variations and modifications are possible. Computer system 1000 canhave other capabilities not specifically described here (e.g., mobilephone, global positioning system (GPS), power management, one or morecameras, various connection ports for connecting external devices oraccessories, etc.). Further, while computer system 1000 is describedwith reference to particular blocks, it is to be understood that theseblocks are defined for convenience of description and are not intendedto imply a particular physical arrangement of component parts. Further,the blocks need not correspond to physically distinct components. Blockscan be configured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how the initialconfiguration is obtained. Embodiments of the present invention can berealized in a variety of apparatus including electronic devicesimplemented using any combination of circuitry and software.

It should be noted that the LERC methods described herein (e.g., methods300, 400, 500, 600, and LERC encoding procedures of FIG. 9A-9E, etc.)can be performed by processing logic that may comprise hardware (e.g.,circuitry, dedicate logic, etc.), software (which as is run on a generalpurpose computing system or a dedicated machine), firmware (embeddedsoftware, or any combination thereof. In some embodiments, the variousmethods described herein can be performed by the processor 1030 of FIG.10. It should also be appreciated that the specific steps illustrated inFIGS. 3-6 provide aspects of a particular method of compression,according to certain embodiment of the present invention. Othersequences of steps may also be performed according to alternativeembodiments. In certain embodiments, the methods may perform theirindividual steps in a different order, at the same time, or any othersequence for a particular application. Moreover, the individual steps ofeach method may include multiple sub-steps that may be performed invarious sequences as appropriate to the individual step. Furthermore,additional steps may be added or removed depending on the particularapplications. One of ordinary skill in the art would recognize andappreciate many variation, modification, and alternatives of the method.

The above disclosure provides examples and aspects relating to variousembodiments within the scope of claims, appended hereto or later addedin accordance with applicable law. However, these examples are notlimiting as to how any disclosed aspect may be implemented,

All the features disclosed in this specification (including anyaccompanying claims, abstract, and drawings) can be replaced byalternative features serving the same, equivalent or similar purpose,unless expressly stated otherwise. Thus, unless expressly statedotherwise, each feature disclosed is one example only of a genericseries of equivalent or similar features.

Any element in a claim that does not explicitly state “means for”performing a specified function, or “step for” performing a specificfunction, is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. §112, sixth paragraph. In particular, the use of“step of” in the claims herein is not intended to invoke the provisionsof 35 U.S.C. §112, sixth paragraph.

What is claimed is:
 1. A compression method comprising: receiving pixeldata representing a two dimensional array of pixels, wherein each pixelis associated with a data value; receiving a first user definedparameter defining a maximum error allowable per pixel for a compressionprocessing routine; executing the compression processing routine,wherein the data value for each pixel is compressed using thecompression processing routine, and wherein an error caused by thecompression processing routine is equal to or less than the maximumerror allowable; and encoding the pixel data based on the compressionprocessing routine.
 2. The method of claim 1 wherein a pixel depth ofthe pixel data is larger than 8 bit.
 3. The method of claim 2 whereinthe pixel data is at least one of scientific data, elevation data, orsatellite imagery data.
 4. The method of claim 1 wherein the compressionprocessing routine is a no transform compression processing routine. 5.The method of claim 4 wherein the compression processing routinecomprises quantizing the pixel data into an array of unsigned integervalues.
 6. The method of claim 5 wherein the compression processingroutine further comprises bit stuffing the array of unsigned integersinto a byte array.
 7. The method of claim 6 wherein the compressionprocessing routine further comprises dividing the 2D array of pixelsinto pixel blocks, wherein encoding the pixel data further includesencoding each of the pixel blocks.
 8. The method of claim 7 whereindividing the 2D array of pixels into pixel blocks further comprises:determining a set of statistics from each block, wherein the set ofstatistics includes a maximum and minimum value in each block and anumber of valid pixels in each block; and computing the number of bytesrequired to maximally compress each of the blocks based on the set ofstatistics.
 9. The method of claim 7 wherein the encoding each of theblocks occurs only once.
 10. A non-transitory computer-readable storagemedium comprising a plurality of computer-readable instructions tangiblyembodied on the computer-readable storage medium, which, when executedby a data processor, provides a lossy compression method, the pluralityof instructions comprising: instructions that cause the data processorto receive pixel data representing a two dimensional array of pixels,wherein each pixel is associated with a data value; instructions thatcause the data processor to receive a first user defined parameterdefining a maximum error allowable per pixel for a compressionprocessing routine; instructions that cause the data processor toexecute the compression processing routine, wherein the data value foreach pixel is compressed using the compression processing routine, andwherein an error caused by the compression processing routine is equalto or less than the maximum error allowable; and instructions that causethe data processor to encode the pixel data based on the compressionprocessing routine.
 11. The non-transitory computer-readable storagemedium of claim 10 wherein a pixel depth of the pixel data is largerthan 8 bits.
 12. The non-transitory computer-readable storage medium ofclaim 11 wherein the pixel data is at least one of scientific data,elevation data, or satellite imagery data.
 13. The non-transitorycomputer-readable storage medium of claim 10 wherein the compressionprocessing routine is a no transform compression processing routine. 14.The non-transitory computer-readable storage medium of claim 13 whereinthe compression processing routine comprises quantizing the pixel datainto an array of unsigned integer values.
 15. The non-transitorycomputer-readable storage medium of claim 14 wherein the compressionprocessing routine further comprises bit stuffing the array of unsignedintegers into a byte array.
 16. The non-transitory computer-readablestorage medium of claim 14 wherein the compression processing routinefurther comprises dividing the 2D array of pixels into pixel blocks,wherein encoding the pixel data further includes encoding each of thepixel blocks.
 17. The non-transitory computer-readable storage medium ofclaim 16 wherein dividing the 2D array of pixels into pixel blocksfurther comprises: instructions that cause the data processor todetermine a set of statistics from each block, wherein the set ofstatistics includes a maximum and minimum value in each block and anumber of valid pixels in each block; and instructions that cause thedata processor to compute the number of bytes required to maximallycompress each of the blocks based on the set of statistics.
 18. Thenon-transitory computer-readable storage medium of claim 16 wherein theencoding each of the blocks occurs only once.